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PERFORMANCE MEASUREMENT FOR MUNICIPALITIES 


Summary 


The current economic climate is having a dramatic impact on all municipal govern- 
ments. Demands for services and their costs are increasing at a faster rate than the 
growth in assessment and transfer payments from senior levels of government. 
Some municipalities are rising to the challenge by adopting new management 
systems and budgeting techniques with a view to increasing productivity in the 
provision of municipal services and to allocating limited resources for maximum 
benefits. The magnitude of municipal expenditures makes any such attempt worth- 
while. For example, if all municipalities in Ontario were to improve productivity by 
five per cent, an annual benefit of over 225 million dollars would be realized. 


The most progressive municipalities are now recognizing that an essential in- 
gredient in an improved management system is a factual evaluation of efficiency 
and effectiveness in the services provided. Performance measurement fills this 
need. Municipalities are now being challenged to justify every dollar spent. The 
critical data that flows from good performance measurement will facilitate an 
assessment of the decisions taken and a critical evaluation of the operations that 
follow from those decisions. 


This publication first discusses in Chapter | performance measurement in the con- 
text of efficiency and effectiveness, and how it can assist elected officials in 
making essential strategic policy decisions, and aid appointed officials in making 
the tactical operational decisions to implement those policies. Performance 
measurement is therefore a tool to assist good decision-making by managers at all 
levels. 


Approaches to developing a set of performance measurements are outlined in 
Chapter II and criteria for selection of measurements are set out. 


Chapter Ill is the key chapter and describes how a municipal performance 
measurement program can be organized and implemented. The elements are: 


Top management approval and commitment. 
Establishment of a central implementation authority. 
Creation of a project team. 

Selection of the services to be addressed. 
Orientation of all participants. 

Establishment of an implementation plan. 


Oe 


Chapter IV discusses in depth the critical process of collection and use of per- 
formance data. 


Chapter V presents examples of specific performance measurements relating to 
five municipal services: garbage collection, fire, police, inspections and road 
maintenance. 


Performance measurement is not just a theoretical idea. It has been implemented 
in some United States municipalities and Chapter VI describes the experiences of 
Sunnyvale, California and Dayton, Ohio. The City of Thunder Bay, Ontario is moving 
to the adoption of performance measurement as is the Regional Municipality of 
Ottawa-Carleton; their experiences are described, and the Performance Measure- 
ment Manual and Catalogue prepared by the City of Thunder Bay accompanies this 
publication. 


In conclusion, Chapter VII indicates that it is likely that the current cost/revenue 
squeeze being experienced by all governments in Ontario will, in due course, force 
all municipalities to adopt Some form of performance evaluation. We hope this 
publication will provide a good basis to facilitate this, with a resultant improve- 
ment in municipal productivity. 
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I. INTRODUCTION 


This publication is about performance measurement in 
municipalities. Within the context of municipal government, performance 
measurement can be defined as the quantitative determination of the 
effectiveness and efficiency of municipal services. 


Every organization exists for a purpose which is articulated through 
the organization's goals and objectives. The extent to which the 
organization achieves those goals and objectives determines its effective- 
ness. 


Efficiency refers to the relationship between inputs and outputs, 
that is, between the amount of resources used and the amount of product 
or service produced. An organization is said to be efficient if, for a given 
amount of product or service, it minimizes the resourees it uses to 
achieve that output. 


The concepts of effectiveness and efficiency are broad in scope. 
They are applicable to all organizations whether public or private, large 
or small. They apply to the entire organization and they apply to the 
departments or sub-units of the organization. They can have a short-term 
or long-term focus and they can apply to the performance of any task 
whether productive or administrative. 


Implicit in the definitions of effectiveness and efficiency is the 
notion of quantification. Any decision or evaluation regarding a municipal 
service will be influenced by both subjective judgements and objective 
documentation. In the political environment of municipal government 
subjective judgements are indispensible. But any decision or evaluation is 
stronger when supported by factual evidence. Performance measurement 
is a means for improving decisions through a quantitative evaluation of 
effectiveness and efficiency. 


Performance measurement of municipal services can be a valuable 
decision-making tool for elected and appointed officials alike. However, 
the information needs of the two groups, and therefore the measurements 
they use, may be different. For elected officials performance measure- 
ment can help them determine whether municipal services are effective 
in satisfying the demands of the public. It can also assist them in 
efficiently allocating scarce resources among services. For example, 
through a survey of citizen opinion, council may determine that citizens 
are satisfied with service A and service B, but dissatisfied with service C. 
Alerted to this, council can investigate methods of improving the 
effectiveness of service C. Moreover, that same citizen survey may 
reveal that the public ranks service C as more important than A or B. 
Council may therefore decide that a reallocation of funds to C is more 
efficient in raising the general level of citizen satisfaction. 


Performance measurement can assist councillors in making strategic 
policy decisions; it can also help appointed officials in making tactical 
operating decisions. It can indicate whether service delivery is effective 
in terms of the workload undertaken. For example, a council may set as 
one of the municipality's goals a 10% reduction in crimes committed. The 
police chief will develop an array of objectives designed to meet this goal. 
One of these objectives may be to clear 58% of reported crimes by means 


of arrest, on the theory that the incarceration of criminals prevents them 
from committing more crimes, at least during their imprisonment. Thus, 
the police chief would deem his operation to be effective if the crime 
clearance rate via arrest was 58% or greater. He would also wish to exert 
control over the quality of arrest to ensure that the arrest did indeed 
contribute to the criminal's incarceration and that the simple clearance 
rate measure did not lead to a perverse incentive for officers to make 
questionable arrests merely to inerease the clearance rate. Thus, the 
police chief might only count as crimes cleared by arrest, those arrests 
that resulted in conviction. Efficiency could be measured by dividing the 
dollar cost or number of man-hours by the number of arrests or the 
number of crimes investigated. Of course, extraordinary circumstances 
eould arise which might distort performance measurements, For example, 
the apprehension of an extremely dangerous criminal could require a great 
deal of time and resources. The measurements suggested here are meant 
to apply to normal operations. 


To use a further example, council may state that one of the 
municipality's goals is to provide, 99.9% of the time, a continuous and 
adequate supply of potable water which conforms to legislated quality 
guidelines. Councils may have neither the time nor the expertise to 
evaluate performance against a large variety of flow and quality criteria. 
However, performance could be measured for the councils' purpose as the 
percent of days during the year that the stated goal was achieved. 
Appointed officials, on the other hand, would be interested in more 
technical and precise measurements such as the percent of water samples 
meeting bacterial, PH and other quality standards, and the percent of the 
time that water pressure standards were met. 


The discussion and examples presented above are not meant to 
suggest that performance measurement is obvious or easy. Any 
experienced municipal official can see that it is not. Rather, the 
discussion and examples are only meant to indicate that a program of 
performance measurement of municipal services can be of value to both 
elected and appointed officials. The full value of such a program, 
especially for elected officials, would accrue if performance 
measurement were undertaken in all municipal services. But performance 
measurement can also be applied to any individual service. It may in fact 
be desirable to implement such a program on a service-by-service basis, 
attempting to measure the day-to-day operations of services first and 
embarking on a fully integrated program only after appointed officials 
have mastered performance measurement for operating purposes. More 
extensive discussions of the selection of performance measurements, the 
techniques of data collection and the uses of performance information are 
presented later in this publication. 


Both elected and appointed municipal officials must bear in mind 
that performance measurement will not in itself produce higher levels of 
effectiveness and efficiency. These qualities will be achieved only 
through decisions to rearrange resources, work methods and task 
priorities. But performance measurement can provide the hard data 
necessary to redirect effort. A parallel might be drawn between 
municipal performance measurement and the cost accounting, market 
research and quality control systems of private business. If large business 
enterprises see such systems as essential to decision making, why should 
large public enterprises behave differently? 


Clearly, municipal politicians and civie servants have always desired 
to provide effective, efficient services to the public. But often they have 
lacked valid, reliable, objective performance data, and have therefore had 
to rely heavily on political judgement and professional experience. 
Performance measurement will not supplant political judgement and 
professional experience. Rather it will augment them in the decision 
making process. The managers of municipalities, whether they be the 
elected or the appointed officials, make decisions on the basis of the best 
available information. Performance measurement will improve that 
information. 


Performance measurement, therefore, is not offered as a panacea to 
all municipal ills but rather as a component of good management. In fact, 
it will likely find its earliest acceptance and greatest success in those 
municipalities which already exhibit the characteristics of advanced, 
progressive municipal management. Some of those characteristics include 
the Chief Administrative Officer (CAO) form of organization, a 
computer-based information system, and an advanced budgeting system. 
But any municipality or municipal department should be able to introduce 
some degree of performance measurement into its operations; even 
though a municipality may not have ideal management conditions, it will 
have taken a large progressive step in that direction if it implements 
performance measurement. 


This publication is a practical guide to developing a municipal 
performance measurement program. However, the subject of 
performance measurement is both large and complex, and many of the 
topics discussed in this publication are but distillations of current theory 
and practice. The bibliography at the end of this publication sets forth 
some of the major works on the subject of performance measurement and 
is recommended to interested municipal managers. 


Performance measurement is a different, even radical, new 
technique of municipal management and for most municipalities it will 
not be simple to develop and implement. It will create organizational 
stresses as more emphasis is shifted to quantifiable performance criteria, 
it will require time and patience, and it will involve cost in terms of both 
money and manpower. Yet as a growing number of municipalities are 
learning, performance measurement can provide the information that 
managers need to improve municipal effectiveness and efficiency. 
Performance measurement is a feasible management system, having been 
implemented in such U.S municipalities as Sunnyvale and Palo Alto in 
California, Fairfax County in Virginia, Scottsdale in Arizona, Dayton in 
Ohio, Washington, D.C., Dallas, Milwaukee, Nashville, St. Petersburg, 
Cineinnati and many others. A growing number of Canadian 
municipalities are also moving to implement performance measurement, 
among them Thunder Bay, the Region of Ottawa-Carleton, Scarborough 
and Halifax. The experiences of four municipalities are briefly described 
later in this publication, but interested municipal managers are urged to 
investigate in more detail the performance measurement programs of 
municipalities with experience in this field. 


Il DEVELOPING A SET OF PERFORMANCE MEASUREMENTS 


A. Effectiveness and Efficiency in the Municipal Context 
i) Effectiveness 


Effectiveness has been defined in terms of the extent to which an 
organization achieves its goals and objectives. In the municipal context 
goals are the policy statements which guide the direction of the 
municipality usually over a multi-year time frame. Objectives are 
specific statements which describe measurable managed actions that 
contribute to accomplishing a goal within a more restricted time frame. 
In a sense, goals are strategie aims while objectives are tactical aims by 
which goals are achieved. For example, one of a municipality's goals may 
be to reduce property losses due to fires by 20% over the next five years. 
To accomplish that goal, the municipality may formulate an array of more 
specific objectives, some of which may include an increase of 30% in the 
identification of fire hazards through fire inspection services, a reduction 
in response time of 25% and a reduction of 10% in the amount of fire 
spread after the arrival of the first fire fighting unit. 


Note that goals and objectives do not concern the resources used to 
provide services but rather they concern the impact of the services. A 
municipal service is effective if it achieves the desired impact, as 
specified in the goal or objective statement. Measuring the effectiveness 
of a municipal service then, means stating what the service seeks to do 
and evaluating the degree to which it meets its aims. 


The simple definition of effectiveness as the achievement of goals 
and objectives, however, does not take account of complexities within the 
context of municipal government. These complexities, discussed below, 
are demand, quality and difficulty. 


By demand we mean the perceived need for a service and its 
relative priority in the eyes of citizens and their representatives. 
Although this subject is more akin to policy-making than to performance 
measurement of operations, it is still an important element of effective- 
ness measurement, because if effectiveness means the achievement of 
service goals and objectives, then it is the relative demand for various 
services that should cause goals and objectives to be set. Demand might 
be measured by some combination of such techniques as citizen surveys, 
studies of local economie conditions, reports on the number of 
applications, requests or complaints received, expert evaluations such as 
roads needs studies, and so forth. A municipality should attempt to 
measure the relative demand for services because such information will 
help decision- makers, especially councillors, to identify the services 
deemed most important at a given point in time, the type of service 
needed, and the criteria by which the success of a service at meeting 
those needs can be assessed. In the municipal context, effectiveness 
means not only success at doing things, but also doing the right things. 


By quality, we mean that effectiveness measurements must specify 
that goals and objectives, particularly the latter, must be accomplished 
with a stated standard of excellence. For example, a municipality may 
set such objectives as an increase in crimes cleared by arrest, an increase 


in children placed for adoption, or an increase in potholes patched. But 
what if such measurements motivated employees to meet such increases 
at the expense of the quality of the work performed? What if lack of 
adequate evidence caused a charge to be dropped, a child placement 
lasted only two months, or a road patch disintegrated after three weeks? 
In measuring effectiveness, a municipality must not only know that things 
were done, but also that they were done well. 


By difficulty, we take cognizance that in most municipal services 
there can be a high degree of variability in complexity of the incoming 
workload. Some types of crime are more difficult to solve than others, a 
fire in an advanced stage of spread will likely be more difficult to control 
than one which has just started, and soil conditions in some locations may 
make road reconstruction or repair work more arduous than in other 
locations. As far as possible, effectiveness measurements must make 
allowance for difficulty factors by classifying the relative difficulty of 
workload units. 


ii) Efficiency 


Efficiency measurements define the quantitative relationship 
between inputs and outputs. They can be either productivity ratios 
(output divided by input) or unit cost ratios (input divided by output). For 
purposes of illustration, this paper will speak of the latter type 
exclusively, though either is acceptable. An organization is said to have 
become more efficient if its unit cost ratio declines, that is, if it costs 
less to produce each unit. 


a) Inputs 


Any resource can be used as the numerator of the unit cost ratio. 
For most municipal performance measurement purposes inputs will be 
represented by employee time or dollar cost, although such inputs as 
machine time, amount of land, floor space or units of energy may be used 
for special analyses. Employee time is often used in circumstances where 
it is the only significant controllable input or where wages and salaries 
represent the preponderance of dollar cost. Also, employee time does not 
suffer from the distortionary effects that inflation has on dollar cost. 


In many services, the mix of inputs is complex and input measure- 
ment must be in terms of dollar cost. In many municipalities cost 
accounting systems exist which accumulate cost data for specific services 
provided by municipal departments, but some restructuring of cost data 
may well be required for performance measurement purposes. However, 
three special precautions on the use of cost data for efficiency measure- 
ment should be mentioned at this point. 


First, because year-to-year comparison is an important use of 
efficiency measurement data, the distortionary effects of inflation must 
be taken into account. An appropriate price index should therefore be 
selected to adjust current dollars to "constant dollars". 


Second, because comparison among organizational units or different 
municipalities is also an important use of efficiency measurement data, 
care must be taken that what is defined as cost in one unit or municipality 


is the same in the other units or municipalities to which comparisons are 
made. 


Third, because performance measurements are used for decision 
making purposes, the accumulation of cost data should be guided by the 
requirements of the decision in question. For example, if the efficiency 
of garbage collection crews were being measured, then it would not be 
appropriate to allocate to those crews the administration costs of the 
department, which are outside the control of those crews or their 
supervisors. If the entire department was being measured, departmental 
overheads would properly be included but not such general overhead items 
as the costs of the finance department, the mayor's salary and so forth. A 
good rule of thumb might be to accumulate only those costs which ean be 
directly attributed to the unit or activity under eonsideration. For 
overhead costs, separate cost centres should be defined and separate 
performance measurements developed for them. 


b) Outputs 


The denominator of the unit cost ratio, outputs, is the number of 
units of workload accomplished, for example, lane miles of road snow- 
ploughed, crimes investigated, tons of garbage collected, gallons of 
sewage treated, number of clients served, and so forth. Workload counts 
do not always indicate whether the task was accomplished well or not, and 
a decrease in the unit cost ratio by means of increased output but at the 
expense of service effectiveness and quality cannot be called a true 
productivity gain. Therefore, where possible, a municipality should count 
as units of workload only those which meet certain quality standards. 
This prescription may not always be feasible, of course. For example, the 
number of library users served is much easier to determine than the 
number of satisfied users. Nonetheless, a municipality should attempt to 
count those units of output which contribute toward the effectiveness of 
the service being measured. Even in those services where effectiveness 
may be measured by the degree of client or citizen satisfaction, it may be 
possible to develop some sort of efficiency measurement. For example, if 
a 10% increase in spending on street cleaning was accompanied by a 15% 
increase in the level of satisfaction citizens expressed about the 
cleanliness of streets, then an efficiency gain could be said to have 
occurred. 


Two other types of measurements are often covered under the name 
of efficiency measurement. The first, resource utilization measurements, 
indicate the amount of time in which personnel or equipment provide or 
are available to provide service. Thus downtime for equipment repair and 
maintenance or idle time of work erews would be deducted from total 
time and the remainder divided by total time to calculate a resource 
utilization rate. The assumption here is that the higher the rate, the 
higher the efficiency, although this does not say anything about the 
amount of work accomplished or its quality. 


The second, normalized cost measurements, have often been used as 
efficiency indicators but in fact, demonstrate little relationship between 
inputs and workload. Typical examples of such measurements are cost per 
capita, employees per capita and cost per $1000 of equalized assessment. 
Normalized cost measurements may be useful in establishing rough 
benchmarks over time or among municipalities; however, by themselves 
they say nothing about operational efficiency. 


B. The Selection Process 
i) Criteria for Measurement Selection 


A program to measure the performance of municipal services 
generates data which will be used to make decisions. If those decisions 
are to be made with confidence, the effectiveness and efficiency 
measurements selected by a municipality must conform to the following 
criteria: 


= Validity - A measurement must be as precisely as 
possible an indicator of effectiveness or efficiency, and 
it must be appropriate to the objective of the service. 


2 Completeness - One measurement is usually not enough 
to indicate service effectiveness or efficiency. 
Therefore, a set of measurements for a service should be 
selected which should account for all or most of the 
criteria by which the effectiveness and efficiency of a 
service are evaluated. 


= Clarity - A measurement must be understandable. A 
complex, technical measurement may present some 
difficulty to those without an appropriate professional 
background, for example, councillors, although it may be 
useful to operational managers. By the same token, the 
set of measurements must not be so large and complex 
that decision makers become overburdened with data. 
Most desirable may be a set of measurements that 
include some general measurements for policy-makers 
and some more technical measurements for operational 
personnel. 


- Controllability - Although efficiency and effectiveness 
ean be influenced by factors beyond management's 
control, a measurement must deal with phenomena that 
management can at least partially affect. 


= Accuracy - Decision makers must be able to rely on the 
accuracy of performance data. A measurement must be 
selected with reference to whether it is feasible to 
eonfirm the accuracy of reported results. 


= Uniqueness - A measurement must reveal a character- 
istie of the service that no other measurement reveals, 
in order to avoid overlap and duplication of measurement 
results. 


= Consisteney - A _ performance measurement must 
represent the same type of phenomena in all cases and at 
all times, because performance data is useful only in 
comparison, whether the comparison is to performance 
budgets, to previous time periods or to other 
organizations within or outside of the municipality. 


- Cost - In most eases gathering performance data will 
cost time or money. The benefit to be derived from the 
use of the data should exceed the cost of collecting it. 


- Timeliness - It must be possible to collect, process and 
distribute performance data to decision makers within 
the time frame required by the decision. 


ii) Sequential and Concurrent Selection of Goals, Objectives and 


Measurements 


The process of selecting performance measurements can take place 
in two ways. The most obvious way is the sequential method of discerning 
demand, defining goals and objectives and developing performance 
measurements appropriate to those goals and objectives, as described 
earlier in this chapter. This method of selecting performance measure- 
ments is desirable because it imposes the discipline of planning, control 
and feedback on an organization. Government, including municipal 
government, has often been criticized for operating too much in the crisis 
management mode, whereby services are administered in historically 
established ways until a disaster forces an immediate and radical change. 
However, planning and control of municipal services represents the crisis 
avoidance mode of management, while information feedback and the 
resultant revision of plans allows for flexibility in the event of changed 
circumstances. But even more fundamental than the discipline imposed 
through goal and objective formulation, is the notion that no organization 
ean know what performance to measure until it knows what it wants to 
do. Logically, goal and objective formulation should precede the selection 
of performance measurements. This, then, is the theoretically superior 
way to select efficiency and especially effectiveness measurements. 


Theory is one thing; reality is often another. The reality of 
municipalities is that only a handful have a formal statement of measur- 
able goals and objectives. This lack of experience can cause a goal and 
objective setting exercise to procede quite slowly. 


Such was the experience of the City of Thunder Bay, Ontario, when 
it embarked upon its performance measurement program. To deal with 
this problem, the City developed goals and objectives concurrently with 
performance measurement selection. For each service the City 
assembled a large variety of performance measurements with examples 
drawn from the literature on the subject, from the experience of other 
municipalities which had implemented performance measurement 
programs and from the ideas of the City's elected and appointed officials. 
The managers of each service, assisted by a central group of analysts 
responsible for developing the performance measurement program, 
selected those indicators of effectiveness and efficiency which they 
believed to be most appropriate to their particular services. Officials in 
the City of Thunder Bay hold that this concurrent selection of goals, 
objectives and measurements not only speeds up the selection process, but 
also is theoretically acceptable because through the selection of 


performance measurements previously unarticulated goals and objectives 
are revealed. 


Whether a municipality uses the sequential method or the 
concurrent method of selecting effectiveness and efficiency indicators is 
immaterial, provided the end result is a set of performance measurements 
that meet the nine criteria outlined in the previous section of this 
chapter. Essentially, a municipality should use the way that will work 
best in its own situation. 


fl ORGANIZING A PERFORMANCE MEASUREMENT PROGRAM 


The first two chapters of this publication have discussed the theory 
of performance measurement. This chapter is a guide for putting theory 
into practice. There are six steps involved in organizing a municipal 
performance measurement program. This chapter describes the 
organization of a performance measurement program designed eventually 
to include all municipal departments. However, an essentially similar 
process could be followed by a department head wishing to organize 
performance measurement in his/her area of responsibility only. 


A. Obtain Approval in Principle 


The rigorous, quantitative performance measurement described in 
this paper clearly represents a major departure from the traditional way 
in which municipalities have evaluated their services and their employees. 
It will therefore be essential that top management, that is, councillors 
and senior administrative officials, be firmly committed to such a 
program. This commitment will be most readily gained when top 
management already perceives the need for increased efficiency and 
effectiveness and when it already displays a progressive and secure 
attitude with respect to new management techniques. In fact, most 
successful performance measurement programs are initiated by the mayor 
or the Chief Administrative Officer (CAO) of a municipality, or by 
department heads when the program is implemented in a single service 
only. 


But regardless of who initiates the program, a general proposal 
should be written which outlines the need for performance measurement 
and the techniques of implementing the program, including the steps 
outlined in this and the next chapter, and which documents typical 
measurements, the experiences of other municipalities which have 
adopted performance measurement, and the expected benefits. Typical 
costs and a time frame should be included, with suggested alternative 
approaches to obtain the maximum benefit/cost ratio for the program. 
The proposal should be circulated for comment to all the top managers 
who will be part of the program. While it would be unrealistic to expect 
unanimous approval, the formal, written commitment in principle of the 
council, the CAO (if present), and most department heads is essential. 


B. Designate Central Implementation Authority 


After approval in principle is obtained, but before a project team is 
constituted, a central implementation authority should be designated to 
oversee and expedite, in a general sense, the implementation of the 
performance measurement program. This authority must ensure that the 
approved principles of the program proposal are adhered to, and must also 
be prepared to arbitrate conflicts and issue directives to deal with any 
bottlenecks or resistance of some municipal employees. 


_ This authority must be vested in someone, whether the mayor or his 
designate, a steering committee of councillors and/or senior appointed 
officials or the chief administrative officer. Evidence suggests that 
performance measurement is more easily implemented where a 
municipality has a strong CAO, principally because the clear lines of 
accountability allow for quicker, more assertive corporate decision- 
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making. Por municipalities without a CAO, performance measurement is 
still feasible but the need for a clearly identified authority is that much 
stronger. 


(Cr Create Project Team 


A team of analysts from varied disciplines must normally be created 
to implement a performance measurement program. Rarely do 
department managers and employees have the experience or time 
necessary to develop a set of good performance measurements and to set 
up data collection and utilization systems. A team of analysts, however, 
can assist departments in defining goals and objectives, in selecting 
measurements and in testing the feasibility of various collection and 
utilization techniques. Larger municipalities may already have a central 
team of analysts, possibly attached to the Treasurer's or CAO's office, to 
conduct the program, and it may also be possible to second personnel from 
other departments to serve on the team. In smaller municipalities, 
though, it may be necessary to hire such analysts or to use the services of 
consultants experienced in municipal performance measurement. 


Regardless of how the team of analysts is obtained, one individual 
should be designated as program co-ordinator. This individual should be a 
municipal employee who has a broad background of training and 
experience in the municipality, and who enjoys good relationships with 
fellow employees, particularly managers. It will be his task to oversee 
the day-to-day operation of the program, to co-ordinate the efforts of the 
team of analysts and to liase with all levels of management. The program 
co-ordinator must have the clear backing of senior management in order 
to be effective. If the performance measurement program is to 
encompass several municipal services, the program co-ordinator must be 
able to devote his full time to the program. 


D. Select Critical Services 


While a comprehensive performance measurement program 
encompassing all municipal services is desirable, resource limitations may 
constrain a municipality, especially a smaller one, to implement the 
program in certain services only. Therefore, it is important to determine 
which services get priority treatment. 


The program co-ordinator should prepare an initial list of services in 
which performance measurement might be implemented. Ideas can be 
solicited from a variety of sources. Operating departments may suggest 
services which they believe it would be useful to measure. For example, a 
department head may be aware of particular problem areas, he may wish 
to gauge the efficiency of new work methods, or he may wish to 
demonstrate that a service is in need of additional resources. Staff 
departments such as Planning or Treasury may suggest services with 
which they have frequent contact as candidates for performance 
measurement. Elected officials are a major source of suggestions. 
Community groups such as service clubs, ratepayers’ associations and 
special interest groups could be canvassed for ideas. Finally, citizen 
surveys could be undertaken to determine the level of satisfaction with, 
and the relative demand for, various municipal services, the more 
important ones being prime candidates for measurement. 
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After the initial list is prepared the program sibs peers * 
screen the list and recommend to council or the moe me 
implementation authority the services to be measured. If all servic see 
eventually to be included in the performance measurement see ha 
important that those included first are successfully implemente oe 
selection of any particular service, then, should be guided by cer ; 
eriteria. The service must lend itself to relatively easy measuremen'. 
The service must not enjoy such strong community or eouncil support igh 
change is unlikely despite the performance measurement results. fe 
service must be significant enough in terms of cost or impact to merit the 
effort. There must be substantial likelihood of improving performance, 
even for services already perceived to be effective and efficient, so that 
performance measurement is seen as a tool to improve services rather 
than as a means of criticizing and embarrassing departments. The service 
should be approaching some critical decision point in terms an its 
continuance, growth, funding or operational techniques. There should be a 
sufficient availability of data. The cost of implementing performance 
measurement in the service should not be too great. Performance results 
should also be obtainable in a relatively short time frame. Finally, 
performance improvements should yield easily visible benefits. 


12h Orientate Participants 


It is essential that the participants in a municipal performance 
measurement program understand and support its aims and techniques. 
Four groups of people will be affected in most performance measurement 
efforts: senior management such as council and senior administrators, 
middle management, employees, and ratepayers and inhabitants. 


Senior management, responsible for the overall effectiveness and 
efficiency of the municipality, will be primary users of performance data. 
Before other groups, they must be educated in the purpose and use of 
performance measurement, so that they may support the program and 
ensure that it serves the municipality's needs. Training sessions will be 
useful in this respect, but because this group's active support is so 
essential to success of the program, time spent by the project co- 
ordinator on personal contact with the central implementation authority 
and other senior management will be time well spent. 


Middle management, who will be responsible for implementation 
must be encouraged to co-operate in the program, and directives to this 
effect passed down from above will not be enough to secure co-operation 
Training seminars should be held to orientate this group. Middle manage- 
ment should also be directly involved in designing the measurements and 
the data collection procedures. Each department should designate a 
representative to act as liaison with the project team. Most importantly, 
senior management must convey to middle management and employees 
that the performance measurement program is not a witeh-hunt but 
rather is a means to improve municipal operations. 


The orientation of employees is very important because it is the 
who will often provide performance data, because the nature of their 
work may be directly affected by the decisions based on such data and 
beeause the successful implementation of these decisions may b 
dependent upon the co-operation of employees. Through literatur 
seminars, management should clearly communicate to employees ee 
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purpose and structure of the performance measurement program. 
Employee unions should be consulted from the outset. To assuage any 
apprehension on the part of municipal employees, management could 
ensure that no existing employee would be laid off as a direct result of 
the program, though total staff complement could decline through 
attrition. Some municipalities might also wish to involve employees 
directly in service evaluation, as Cincinnati has done. An incentive 
program could also be tied to municipal performance measurement, as has 
been done in Kansas City, Flint in Michigan and Philadelphia. 


It is for the benefit of ratepayers and inhabitants that municipal 
services are provided. Moreover, many effectiveness measurements rely 
on citizen or client evaluation. It is important that the municipality 
encourage ratepayers and inhabitants to become more knowledgable about 
local services. Ratepayers and inhabitants should be informed through 
public announcements that a performance measurement program is being 
initiated and that their perceptions of performance may be solicited 
through surveys. Ratepayers and inhabitants could also be recruited to sit 
on a performance measurement steering committee or advisory council, 
as is done in Washington, D.C., and St. Petersburg. Summaries of 
performance results could be mailed to citizens or published in the local 
press. On this last point, it is quite important to orientate the press to 
the aims and techniques of performance measurement. The press should 
be encouraged to view the program as an honest attempt to improve 
municipal services and not as an opportunity to embarrass local officials. 
Members of the press have a right to publicize consistently poor 
performance, of course. But they also owe it to the community to allow 
performance measurement time to work. 


FY Establish Detailed Implementation Plan 


A detailed implementation plan must be created and approved. This 
plan should outline the major implementation tasks. They are: 
1) identify service goals and objectives; 2) select effectiveness and 
efficiency measurements; 3) establish data collection procedures; and 
4) determine the means by which performance results will be analyzed. 
The first two tasks have been discussed in Chapter II. The latter two are 
detailed in Chapter IV. 


Two other elements, though, are needed in a good implementation 
plan: time and resource requirements, and performance criteria of the 
program itself. 


Time and resource requirements are difficult to estimate because 
the size and scope of performance measurement programs can vary. A 
fully operational performance measurement program seems to require 
some three years from inception, through initial start-up and "de- 
bugging", to an established program that can be used with confidence over 
an extended period. Resources required can be few if the program is 
confined to one or a few services, or large if a municipal-wide program is 
undertaken. Resources which are mentioned at various points in this 
publication could inelude a project team and co-ordinator, consulting 
services, training and orientation costs, redesign of information forms, 
trained observers, citizen surveys and, of course, the time of council, 
managers and employees. Estimates of these resources must be made for 
inclusion in the project proposal as well as the detailed implementation 
plan. They can be based on the needs of the municipality and possibly on 
the experiences of other municipalities. 
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Costs must be weighed against benefits, and the benefits of 
municipal performance measurement, like its costs, are hard to estimate. 
Essentially, the benefit is better information. This information can be 
used by decision makers to improve municipal effectiveness and 
efficiency; in other words, to make more productive use of municipal tax 
dollars. 


A set of performance criteria of the performance measurement 
procram itself is the final element of the implementation plan. If the 
program has achieved its aim of providing better information to 
management, then the following results should be observed. First, 
councillors and staff would be better informed about municipal services, 
and therefore able to improve the quality of strategic and operational 
decisions. Second, many departments should be able to find, implement 
and document effectiveness and efficiency improvements. Third, the 
controllable unit cost of services should become stabilized. Fourth, 
resources should become allocated on the basis of quantifiably 
demonstrable need, and according to the strength of relative demand in 
the community. Fifth, citizen satisfaction with services should become 
higher. 


These benefits of a municipal performance measurement program 
rest on one important assumption; that management will use performance 
data to make decisions that result in change. Like any information 
system, performance measurement in and of itself will not improve 
effectiveness and efficiency. It can, though, indicate where such 
improvements are desirable and possible. Performance measurement will 
provide new information. But the responsibility for acting on that 
information rests, as it always has, with management. 


The important question of how performance data ean be collected 
and used is discussed in the next chapter. 
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IV COLLECTING AND USING PERFORMANCE DATA 


A. Data Collection Techniques 
i) Data Collection by Operational Personnel 


For most municipalities initiating a performance measurement 
program, the principle source of data will be records compiled by 
municipal employees in both staff and line functions. A great deal of 
information is currently gathered in this way, and some of. this 
information ean be adapted to performance measurement. For example, 
many municipalities already collect workload data such as number of 
eases investigated, number of gallons of water distributed, number of 
clients served, number of repairs made, and so forth. Cost data is also 
compiled, sometimes on a service or task basis, sometimes only on a 
departmental basis. As well, a good deal of financial and non-financial 
information is prepared by municipalities to meet the reporting or grant 
application requirements of senior levels of government. Before 
embarking on a performance measurement program, a municipality should 
survey the data it currently collects to discover what can be used for 
performance measurement purposes. However, it will be necessary in 
many eases both to modify existing data collection procedures to adapt 
them to performance measurement, and to collect entirely new data. In 
an ongoing performance measurement program, most of this new or 
modified data will be collected via municipal operational employees. 


It would be too large a task in this publication to list and evaluate 
the information municipalities now collect or might collect about all 
individual services, although examples relating to some services are given 
later. This section is therefore confined to a general discussion on the 
procedure for gathering performance information internally. There are 
four steps involved. 


First, the required data must be identified. This first step is 
actually the measurement selection process. The performance measure- 
ments selected as being appropriate to the goals and objectives of the 
municipality will define the data needed. If field collection is indicated, 
it would be wise to develop a number of alternative measurements as it 
may not prove feasible to collect some kinds of data. It is also important 
to define rigorously the data required to ensure consistency over time and 
among data collectors. Although some revisions are unavoidable, too 
many revisions will obviate comparability and may create frustration and 
resistance among those physically collecting the information. 


Second, data availability must be determined. As mentioned, 
municipalities often already collect data which could be adapted to 
performance measurement. Existing records should be searched to 
discover if the performance data required, or components of it, are 
currently being collected. The data may be stored or collected by more 
than one agency or jurisdiction (e.g. traffic volume and accident 
statistics). In this case, the co-operation of these agencies must be 
obtained through negotiation, offers to provide the physical resources 
needed to collect the data, or as a last resort through directives issued by 
higher authorities. If new data is required, it can be collected either by 
adding more data items to records routinely kept or by establishing new 
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records and procedures. While the latter method may be necessary in 
some eases, the former is more desirable in that it may cause fewer 
disruptions and less confusion among those physically collecting the data. 
Whether the data is new or already exists, it would be wise to pre-test it 
for validity, accuracy, "collectibility" and timeliness to avoid problems 
during actual implementation. 


Third, the data must be physically eollected. Although clerks 
familiar with data accumulation may be the most reliable data collectors, 
it will often only be feasible to collect information from operational 
personnel, that is, those delivering the service being measured. 
Operational personnel should first undergo an orientation program to 
make them aware of the purpose and methodology of the performance 
measurement program and their contribution to it. In this respect their 
participation in selecting measures or at least in commenting on 
collection procedures should be encouraged. Operational personnel will 
need thorough training and clear concise directions on how to complete 
data collection forms. The forms should be pre-printed and include 
instruetions and examples. Periodic re-training sessions and spot checks 
for accuracy will be needed. 


Fourth, accuracy must be verified. This step is not chronological, 
but rather should occur at the same time data is being collected. Two 
major types of data collection errors could occur: clerical errors and 
estimating errors. 


Clerical errors such as transposed digits, arithmetical mistakes and 
incorrectly recorded figures are reasonably easy to detect principally by 
comparison to forms filed by others and by sample audits. If the error 
rate is large, say ten percent, the source of the errors must be isolated. 
If errors are confined to specific individuals, retraining, re-emphasis of 
the importance of accuracy, or replacement may be called for. If errors 
are common to all data collectors, clarification or redesign of forms is 
probably required. 


Estimations are often needed in municipal performance measure- 
ment. For example, a measurement that is often suggested as indicating 
the effectiveness of the fire fighting force is "percentage of fire spread 
after the arrival of the first unit". Such a measurement cannot be 
verified after the fact. Estimation errors can be minimized in the 
following ways. 


First, a rating system must be developed to guide estimation. It 
Should contain clear and comprehensive definitions and should be pre- 
tested. The operational personnel who will be making the estimations 
must be trained and tested in the use of the rating system. Second, 
independent tests or audits could be periodically conducted, although such 
tests may influence results. For example, an independent observer testing 
the measurement cited above would likely have to wait at the fire station 
for an alarm to be turned in, alerting the person being tested to apply 
particular rigour to his estimation. Third, comparative data from other 
units, or possibly other jurisdictions, could be used to gauge accuracy. 
Finally, other more precise measurements might serve as some sort of 
check. In this example, "time to control fire spread" can be more 
rigorously collected and may bear some relationship to "percentage fire 
spread after arrival of the first fire fighting unit". 
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ii) Trained Observer Ratings 


Suppose a municipality set a goal of having streets clean of litter 
and refuse. How would it measure the cleanliness of its streets? Or 
suppose it wanted to provide the motor vehicle users of the community 
with smooth streets free of potholes, bumps and excessive patching. How 
can the municipality determine the relative "smoothness" or "bumpiness" 
of its roads? 


Elected officials often make these evaluations although their judge- 
ments are usually subjective. In many communities the heavy workload of 
councillors and the sheer size of the community may make it imposssible 
for councillors to rate municipal services in any systematic way. It may, 
therefore, become necessary to delegate the responsibility for observing 
and rating municipal services. Because it would probably be unfair to ask 
those employees providing a service to rate subjectively their own 
performance, a municipality may wish to employ independent observers 
who are trained in rating the performance of municipal services. 


The use of trained observers to measure municipal performance is 
not widespread at this time. Nevertheless, a number of large cities in 
North America, including Washington, D.C., Nashville, and St. Petersburg 
have been using trained observer ratings for as long as five years now. 
The use of trained observers has enabled these cities to develop 
performance measurements which are more technically precise than those 
possible from citizen or user surveys, and which could not be derived from 
numerical data gathered by operational personnel. But there are obvious 
costs involved in using this technique of performance data collection, and 
the value of the technique has yet to be demonstrated in a Canadian 
setting. We therefore recommend that any municipality considering using 
trained observer ratings should carefully study the procedures used in the 
above-noted cities. We can, however, provide an overview of trained 
observer rating procedures. 


Observers must be provided with explicit definitions for the grades 
of each condition to be measured in order to minimize perceptual biases. 
Thus, definitions should be both written and visual, i.e. photographs of 
typical scenes exemplifying various grades of performance should be 
provided. The definitions should also be explicit in terms of the area to 
be inspected, the factors that determine various grades and the precise 
conditions that are to be rated. There should not be too many degrees of 
performance (four or five would do, for example, 'poor, average, good, 
excellent" or "smooth, moderately bumpy, very bumpy, safety hazard"). 


Procedures must be instituted for selecting inspection locations, for 
directing observers and for recording and processing the data. To 
minimize the cost of this technique, it may be desirable to observe only a 
sample of locations rather than all locations. If so, all locations must be 
identified and an appropriate sample selected. Supervisory staff should be 
appointed to schedule observers, retrieve the data from them and 
generally direct their activities. The rating data should be recorded on 
pre-printed forms and transcribed onto summary sheets soon after the 
ratings have been conducted. The transcription could be done by clerical 
staff, although some municipalities assign this task to observers on a 
rotating basis. 
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Observers must be selected and trained. If proper definitions have 
been formulated and data recording forms are clear and comprehensive, 
then the inspection task should not require personnel highly qualified ina 
technical sense. But the observers should be reliable, able to withstand a 
certain degree of tedium, able to work independently, and attentive of 
detail. Students might be good and relatively inexpensive observers. 
Observers must be thoroughly trained. They should be tested individually 
to evaluate their understanding of the system, and they should be tested 
as a group to ensure that the written and photographie definitions result 
in little rating variation among different observers. 


Observer ratings must be monitored and verified. Even when the 
variety of observed conditions does not change, people tend to compress 
their ratings toward the middle of a scale as time passes. To combat this, 
it will be necessary for supervisory staff to check ratings on a random 
basis. Such checks must of course be made soon after an observer's 
inspection, and it should be made known to observers that such monitoring 
will take place. 


iii) Citizen/Client Surveys 


The ultimate test of the effectiveness of a municipal service is the 
satisfaction of the citizens with that service. Municipal elections are, in 
a sense, a regular measure of the citizenry's level of satisfaction. But 
there are major drawbacks to relying solely on the electoral process to 
measure the effectiveness of municipal programs. For one thing, two or 
three years, the typical term of municipal office in Canada, is a long time 
for a department to wait to find out whether it is doing a good job. More 
importantly, elections are a "noisy" mechanism for evaluating programs. 
All services are being evaluated at once, and a number of "extraneous" 
factors - personalities, economic conditions, party affiliations, ete. - can 
enter into the equation. 


One answer may be regular surveys of citizens' attitudes toward the 
municipality and its services. A number of U.S. cities, including Dayton, 
Ohio, Washington, D.C., St. Petersburg, and Nashville have conducted 
such surveys. The City of Toronto is also experimenting with surveys. 


A survey is carried out by administering a questionnaire to a 
scientifically selected sample of the population. The survey can deal with 
a particular service of the municipality or some or all of the services. 
Questionnaires should generally try to elicit three types of information. 
The first is the use that a citizen makes of a service; the second is his or 
her evaluation of the service; and the third is certain personal character- 
isties (residence, income level, ete.). 


Few municipalities will have the resources to design and implement 
surveys on their own. Even if there is in-house expertise for the designing 
of surveys, the use of existing personnel for interviewing can interfere 
with the performance of regular duties. For this reason, it is usually best 
to engage the services of a market survey or opinion-polling firm. The 
cost will have to be weighed very carefully against the expected benefits. 


For several reasons, the citizen survey technique of municipal 
performance measurement is probably of more value for making strategic 
decisions than for evaluating the performance of management in an 
operational sense. First, citizens often exhibit a low level of interest in, 
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and knowledge about, the daily operation of municipal government. It is, 
of course, valid for citizens to evaluate services however they may have 
arrived at their opinions. But if their opinions are impressionistic rather 
than knowledge-based, the value of citizen surveys is probably greater in 
evaluating the past and future community goals in an overall sense than in 
evaluating the operating objectives of municipal departments. 


Second, the evaluation of services often results in overall ratings at 
the middle of the scale (e.g. "fair to good"). This phenomenon is likely a 
pesult of lack of knowledge or interest. Here again, the usefulness of such 
surveys in measuring performance against objectives is questionable. 
However, if a service received an extreme rating, this fact, juxtaposed 
against the tendency to compress ratings, would single that service out 
for special attention. 


Third, a citizen survey can be costly. One U.S. study estimated in 
1975 that multi-service citizen surveys could cost between $10 and $15 
per interview for surveys of between 600 to 1000 interviews. The City of 
Toronto's survey of 1200 citizens, conducted in the summer of 1981, cost 
over $50,000. Many municipalities may be willing to make this invest- 
ment only on a needs basis, possibly for use as a budgetary planning tool. 


Client surveys can be a much richer source of information on the 
operating performance of municipal services. They can be designed in 
‘much the same way as general citizen surveys but can be administered on 
site to those using the service. Thus, a municipality may be more 
confident that these survey respondents do have some knowledge of the 
service they are using. A drawback to client surveys is that they do not 
seek data from those citizens who may be dissatisfied and therefore do 
not use the service. 


Be Using Performance Data 


We have already stated that today's difficult social and economic 
climate is challenging municipal governments to improve the effective- 
ness and efficiency of the services they deliver. The purpose of this 
publication is to provide municipal governments with a tool to assess the 
extent to which they are coping with this challenge. Municipal 
performance measurement cannot, in and of itself, improve effectiveness 
and efficiency. But it can indicate where performance is superior, 
satisfactory or inferior. Supplied with this information, elected and 
appointed officials can investigate the causes for varying performance 
levels and can take decisions to ensure that superior performance is 
emulated, that satisfactory performance is maintained and that inferior 
performance is improved. 


Judgements about superior, satisfactory and inferior performance 
can be made only through comparison. There are five ways of making 
comparisons. 


First, actual performance can be compared to performance 

standards. Municipal financial officers will be familiar with standard 
costing techniques often used in private sector manufacturing enterprises, 
by which a standard amount of material and equipment and labour time to 
produce a unit of output is determined, often by industrial engineering 
methods. Performance ean be appraised by comparing what did happen to 
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what should have happened, and can be further investigated through 
variance analysis. It may be possible to develop similar performance 
standards for municipal services, particularly in such engineering-based 
services as water supply and distribution, solid and liquid waste collection 
and disposal, and road maintenance. However, few standards currently 
exist even in these services, let alone in the so-called "soft" services such 
as the social services. A notable exception is the Municipal Maintenance 
Management System (MMMS) developed in 1971 by the Ontario Ministry of 
Transportation and Communications for municipal road maintenance 
services. This system, which employs classical standard costing concepts, 
is in use by many municipal governments throughout North America, and 
has been extended by some municipalities beyond road maintenance into 
many other physical services. Although the use of municipal performance 
standards is not widespread at this time, MMMS is proof that it is both 
possible and very useful to develop standards as criteria against which to 
measure performance. 


Second, performance can be compared over time. Performance 
during a previous time period can serve as a benchmark against which to 
evaluate current performance. Trend analysis is also useful to assess 
effectiveness and efficiency vis-a-vis long-term goals and objectives. As 
mentioned earlier, comparisons of performance over time must take 
account of the distortionary effects of non-controllable factors such as 
inflation. 


Third, the performance of organizational units can be compared. In 
situations where a service is being delivered by more than one 
organizational unit doing essentially similar tasks, (e.g. garbage collection 
erews, day-care centres, fire stations), it may be possible to measure 
relative effectiveness and efficiency and to compare the performance of 
individual units to the norm. 


Fourth, one municipality's performance can be compared to the 
performance of other municipalities. This sort of comparison ean be 
useful in establishing norms or acceptable ranges of performance. There 
is also a high probability that investigation of effectiveness and efficiency 
differences will reveal superior operating methods and technologies which 
could be transferred among municipalities. However, care should be 
taken that proper comparisons are made. For example, municipalities 
may employ different performance measurements, they may collect data 
in different ways, and they may experience different situations which 
render comparisons inappropriate. Despite these limitations, 
intermunicipal comparison is a popular performance measurement 
technique among many municipalities and is being addressed by senior 
government levels as well. For example, the Province of Ontario is 
engaged in a project to develop comparative unit cost measurements for a 
range of municipal services. 


Fifth, actual performance can be compared to planned performance. 
Comparison of actual performance against goals and objectives may be 
the most important use of performance data for management. If such 
comparison is made early enough, it can assist managers in controlling 
operations, that is, in identifying problem areas and taking corrective 
action. It can also assist managers in revising old plans and formulating 
new ones. Moreover, this type of comparison is the most relevant to 
management because it deals with the organization's own situation, its 
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own capabilities and its own desires, and because it deals with the present 
and the future, the only time frames over which management can have 
any influence. 
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Vv EXAMPLES OF PERFORMANCE MEASUREMENTS 


This Chapter presents specific performance measurements for five 
municipal services. It would be impossible in a publication of this size to 
compile measurement examples for all municipal services. — As well, 
because measurements relate to objectives, different objectives in 
different municipalities will require different performance measurements, 
and thus no claim is made as to the general applicability of measurements 
presented in this chapter. Readers who would like more examples can 
refer to Thunder Bay's Performance Measurement Manual and Catalogue 
which accompanies this publication and to the publication Hatry, et al, 


How Effective Are Your Community Services?, referenced in the 
Bibliography. 


A. Garbage and Rubbish Collection 


This service has drawn attention in performance measurement 
because it is highly visible and deals with easily measurable services. The 
tasks of employees involved in this work ean be categorized, and the work 
is repetitious in nature thus allowing for consistency of evaluation. Also, 
the workplace (or route) remains relatively constant. The workload can 
be measured or weighed with reasonable accuracy and the necessary 
equipment and supplies do not vary appreciably over time. 


In the past ten years there have been considerable changes in 
collection practices with the introduction of new packer equipment such 
as the mechanical arm one-man loader. Computerized routing systems 
have been developed which make optimal use of men and equipment. 
Even without these two innovations crew sizes have been reduced from 4 
or 5 per truck to 3 and sometimes 2 men per truck. Clearly, opportunities 
for productivity increases exist in this service. Performance measurem- 
ent can help identify and evaluate these opportunities. 


Department - Public Works 

Program Area - Garbage and Rubbish Collection 

Objective: To collect 100% of all garbage and rubbish on all designated 
routes according to schedule with a minimum of. eitizen 
inconvenience at an average cost of $X per ton. 

Measurement Data Sources 

Effectiveness 

1) percent of garbage collected - trained observer ratings 

2) percent of routes completed - trained observer ratings, time 
on schedule sheets, citizen complaint records 

re: late pick-ups 
3) percent of citizens satisfied - citizen survey 
4) number of complaints re: - citizen complaint reeords 


missed collection, noise, 
spillage, damage, etc. 
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Efficiency 


1) cost per ton -cost accounting records for cost; 


weigh scales for tonnage 
= when weigh scales lacking: 


2) cost per pick-up - internal records 
3) cost per residential - internal records 
(or commercial) customer 
served 


B. Fire Fighting 


Although fire fighting accounts for a large percentage of the 
municipal budget, performance measurement has rarely in the past been 
considered for this highly visible and vital service. However, significant 
technological advances have been made in recent years which could 
present opportunities for performance improvement. As well, today's 
financial troubles are forcing municipalities to reassess the growth in 
staffing levels and the man/machine resource mix. Most municipalities 
which may face cost saving decisions in fire fighting though, will wish to 
evaluate such decisions in terms of the continued effectiveness of the 
service. Performance measurement can greatly facilitate making these 
decisions. 


Department - Fire 

Program Area - Fire Suppression 

Objective: To reduce, on a per fire basis, fire losses to persons and 
property to X% of previous year's losses through prompt fire 


suppression activities at an average cost of $X per alarm. 


(Note: Only count fires not out on arrival of first unit, and classify 
fires by size on arrival and by type of occupancy). 


Measurement Data Sources 
Effectiveness 


1) number of civilian casualties - internal records 
per fire. 


2) number of firefighter - internal records 
casualties per fire. 


3) dollar loss per fire - estimations or insurance records. 
4) dollar loss as a percent - insurance records - i.e. 
of potential loss insured loss divided by insured 
value 
5) percent of fires in which - internal records 


spread after arrival is 
eonfined to X%. 
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6) average time to control fire - internal records 


7) average response time - internal records 
Efficiency 

1) cost per alarm - internal records 
2) ost per fire - internal records 


Ce Criminal Investigation and Apprehension 


Policing is another highly visible service about which the public is 
most sensitive. Like fire suppression, new technology has presented some 
opportunities to police departments, yet the labour intensive cost 
structure of policing and the variable nature of its workload still renders 
eosts difficult to control and performance measurements difficult to 
conceptualize. Here, too, effectiveness is of paramount importance 
because even though many factors determine whether a community is a 
safe place in which to live and work, the effectiveness of police programs 
is an extremely important factor. Here we deal with but one program 
area of police work, criminal investigation and apprehension. 


Department - Police 

Program Area - Criminal Investigation and Apprehension 

Objective: To investigate reported or discovered crimes promptly and 
to bring X% of offenders promptly to justice at an average 


cost of X man-hours per investigation. 


(Note: Crimes should be classified by type) 


Measurement Data Source 
Effectiveness 
1) percentage of emergency or - internal records 


high priority calls responded 

to within X minutes, and per- 
centage of non-emergency calls 
responded to within Y minutes. 


2) percentage of crimes cleared - internal and court records 
by arrest which result in 
conviction. 


3) percentage of cases cleared - internal records 
within X days. 


4) percent of citizens victimized - citizen survey 
by erime who expressed 
satisfaction with police 
investigation efforts. 
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Efficiency 


1) Man-hours per investigation  - internal records 


1D)., Building and Plumbing Inspections 


Inspection services are being challenged by the same fiscal squeeze 
that has affected other municipal services. But one of the problems with 
inspection services is the inability to reduce costs substantially through 
mechanization or other technological methods. Inspections must be 
conducted by trained technicians, foreing any productivity improvements 
to come from reorganization of human resources or adjustments in 
inspection procedures. Performance measurement will be needed to 
evaluate such reorganization and adjustment. Performance measurement 
is feasible in building and plumbing inspection services because tasks are 
fairly routine and can be categorized with relative rigour. Also, standards 
for each item being inspected are usually set by law which allows for a 
standardization of performance at least within a _ given 
community. 


Department - Engineering 
Program Area - Building and Plumbing Inspection 
Objective: To provide prompt inspections to uphold legislated building 


and plumbing requirements at an average cost of X man- 
hours per inspection. 


Measurement Data Source 
Effectiveness 
1) percentage of inspections - internal records 
conducted within X days of 
request. 
2) percentage of periodic - internal records 


inspections conducted 
according to schedule. 


3) accuracy rate of inspections - follow-up inspections made by 
supervisory staff soon after initial 
inspection-sample basis only. 


4) percentage of violations - internal records 
corrected. 
5) percentage of court actions - internal and court records 


processed where judgement 
favours municipality. 


Efficiency 


1) man-hours per inspection - internal records 
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E. Loose-Top Road Maintenance 


Performance measurement should be feasible in most road 
maintenance services because they usually have specifically measurable 
work standards and materials. However, pre-existing road conditions and 
quality of the work completed are very important determinants of 
performance and must be taken into consideration. Therefore, while it is 
important to determine how many man-hours and how much material were 
used to maintain a given section of road it is equally important to know 
the conditions of the road before and after a task was completed. Tasks 
must be classified, then, by difficulty and quality of work completed. 


Technology is providing some improvements in road maintenance, 
such as newly-developed materials and equipment, but staffing is still a 
major cost which needs expert management. Some work crews are often 
more productive than others and their techniques might be transferrable. 
Performance measurement should operate in this mode, improvement 
transfers, rather than in a mode which seeks to embarrass less effective 
and efficient work crews. 


Department - Public Works 

Program Area- Loose-Top Road Maintenance 

Objective - To provide scheduled maintenance through grading and 
repair of loose-top roads to ensure smooth riding 


surfaces at an average cost of $X per km.(grading) and 
$X per ton of gravel (repairs). 


Measurement Data Source 
Effectiveness 
1) percent of grading and - internal records 


repair activities completed 
on schedule, subject to 
road and weather conditions. 


2) relative smoothness of riding - trained observer ratings 
surfaces. 


3) percentage of citizens - citizen survey 


satisfied with condition of 
loose-top roads. 


Efficiency 
1) cost per kilometer of road (grading)- internal records 
2) cost per ton of gravel (repairs) - internal records (note: cost includes 


labour, materials and machine 
time.) 
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VI PERFORMANCE MEASUREMENT IN FOUR MUNICIPALITIES 


Performance measurement of municipal services is not just a 
theory. It is a reality in a growing number of municipalities. In this 
chapter the experiences with performance measurement in four 
municipalities are briefly discussed. Except where otherwise noted, the 
opinions expressed in this chapter are those of the authors. 


Any community considering a performance measurement program 
should carefully study the experiences of other municipalities which have 
broken this new ground. This publication cannot do complete justice to 
the performance measurement systems in the communities discussed in 
this chapter: each system is far too elaborate to allow for a complete 
discussion. Still, these examples indicate that performance measurement 
has been implemented in various ways and with various degrees of 
success. 


A. Sunnyvale, California 


Sunnyvale, a city of 107,000 people lying 40 miles south of San 
‘Francisco, first became involved in performance measurement in 1973 as 
part of a pilot project sponsored by the International City Management 
Association and the General Accounting Office of the U.S federal 
government. The program was initiated by the City Manager and 
extended by his successors. At the time of initiation, Sunnyvale was 
already involved in program budgeting and had a computer based manage- 
ment information system. 


Sunnyvale's performance measurement program is part of an 
elaborate information system which integrates the budgeting, accounting, 
control and auditing functions. Besides strengthening these functions, 
performance measurement data in various degrees of detail can be 
provided to all levels of management, from council for policy planning 
purposes to supervisors for operational analysis purposes. Performance 
measurement is also linked to a personnel evaluation program. 


Performance measurement in Sunnyvale is built around a complex 
hierarchy of responsibility centres through which both cost and 
performance data can be accumulated. Appropriate responsibility centres 
are linked through a numerical coding system which is efficient for 
electronic data processing purposes. 


Performance measurement in Sunnyvale is predicated upon the 
formulation of goals and objectives. Goals, which describe a general 
community condition that the City wishes to achieve or maintain, are 
formulated for each major service area, such as mass transit, law 
enforcement, recreation, and so forth. Goals must conform to policies set 
forth in the City's comprehensive, long-range plan, and thus they are also 
broad in scope and have extended time frames. However, the achieve- 
ment of goals is measured statistically mainly through internally 
generated data on the community conditions which are the objects of the 
goals and occasionally through citizen surveys. Citizen task forces assist 
in goal formulation. 


Within each major service area are grouped a number of operating 
programs. For each of these is developed an array of objectives which 
support City goals. Objectives, (usually of a more short-term nature than 
goals) must describe in specific and measurable terms the results which a 
program is expected to achieve in a stated time frame. 
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Because both goals and objectives in Sunnyvale are statistically 
measurable, performance measurements are simply the quantifiable 
expressions of them, and effectiveness is determined by the extent to 
which goals and objectives are achieved. 


Each of the City's programs is further subdivided into the specific 
tasks which must be accomplished to achieve the program objectives. 
Tasks must be measurable in terms of some specific unit of production or 
service. Task performance is measured against production or service 
targets. As well, it is at this level of operations that efficiency is 
measured in Sunnyvale, by calculating the unit cost of the output of tasks. 


Sunnyvale's performance measurement program was largely 
developed "in-house", although a program co-ordinator was hired to 
administer the system and professional consulting assistance was used in 
the development of computer-based information systems. The City 
Manager provides the central authority for the program, and according to 
Sunnyvale officials the support of this chief executive is definitely the 
most important factor contributing to the system's success. 


Municipal councillors and administrators alike in Sunnyvale are 
highly satisfied with their performance measurement system. They 
measure its success by pointing to an increased understanding by both 
appointed officials and council of the City's operations, a high measured 
level of citizen satisfaction, unit costs which have declined over time and 
that are substantially lower than those of similar Californian cities, an 
increased ability to attract and keep good staff, and a vote by Sunnyvale 
citizens to reject Proposition 13. 


Sunnyvale officials offer the following advice to municipalities 
considering performance measurement. 


(1) A municipality must prepare for resistance from both council 
members and staff because of the initial cost of putting in new 
systems as well as the fear of changing from the more familiar line- 
by-line budgeting system. Orientation of managers from line 
supervisors to department heads is an essential early step in 
implementation. 


(2) The measurements should be kept as simple as possible. 


(3) Strict definitions of efficiency and effectiveness are important, but 
some services defy precise measurement. Therefore, indirect 
measurements and political and professional judgement will remain 
as essential components of a performance measurement program. 


(4) Care should be taken in structuring responsibility centres so that 


flexibility to shift manpower and resources as conditions and needs 
change is not lost. 


(5) Outside consulting assistance will likely be required, but a 
muncipality should develop in-house expertise to ensure both quick 
and accurate program development and to provide continuous 
management of the performance measurement program. 


(6) Time and quality are two key measures of performance. Reducing 


cost without negatively impacting on time of response or quality of 
service is the hallmark of a successful system. 
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B. Dayton, Ohio 


Dayton, with a population of about 200,000, has been consistently 
ranked among the leaders in the development of progressive municipal 
systems. It was one of the first cities to adopt the council-manager form 
of government and it receives continuing recognition in the fields of 
planning, budgeting and citizen participation. Ironically, the most recent 
accomplishment, in the field of performance measurement, came about 
because city services were deteriorating and the City's financial position 
was worsening. A new City Manager was hired in 1974 to reverse this 
trend and one of the key elements in his program of reform was 
performance contracting, a new direction for the management team based 
on close accountability for the performance of each department head. 


At the heart of Dayton's system is a personal contract between the 
City Manager and each department head which specifically sets forth 
factual, measurable objectives upon which the department head's 
performance will be determined during the year. The objectives are 
developed by the department head and submitted to the City Manager at 
the beginning of a formal negotiating process. The objectives finally 
agreed upon are reviewed at the end of each quarter to assess 
performance and also to determine the department head's rate of pay for 
the coming year. 


Dayton's performance measurement effort is essentially a Manage- 
ment By Objectives (MBO) program. Although MBO was attempted prior 
to 1974, without the performance contracting and the factual 
performance measurement aspects, it could not be_ effectively 
implemented. Accordingly, one of the first steps taken by the new 
manager in 1974 was to develop specific and quantifiable objectives for 
each department representing the service levels attainable with current 
budget allocations. Objective setting in Dayton began as, and remains, a 
participative process. Managers at all levels underwent extensive training 
in MBO, and after three years all departments had been included in the 
program. Departmental objectives are incorporated into the annual 
budget. Performance measurements in Dayton, then, measure the extent 
to which objectives are achieved. 


Many of the specific departmental objectives are based on 
effectiveness data gathered in citizen surveys. For example, within the 
police department's crime prevention and control programme, one 
performance measurement was developed as follows: 

Programme: Crime prevention and control. 

Programme Purpose: To prevent and control crime by eliminating the 
opprtunity for criminal behaviour and by the immediate intervention in 
potential criminal activity. 

Activity: Neighbourhood patrol. 


Objective: To increase by three per cent the level of perceived security 
among city residents, as measured by a public opinion survey. 


Performance criteria: Per cent of public opinion survey respondents who 
classify neighbourhoods as "safe anytime" or "safe if careful". 
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There are a number of additional objectives and performance 
eriteria based on a wide range of data, for example, crime statistics, as 
well as the citizen surveys. Each department has a number of 
performance objectives which range from twenty in the Planning 
Department to over 200 specific objectives in the Department of Human 
Resources. 


Dayton's performance measurement system is concerned primarily 
with effectiveness measurement. This focus is due to severe expenditure 
constraints, mainly in staffing, imposed by council. With a fixed level of 
expenditure then, an increase in effectiveness is by definition an increase 
in efficiency. However, this focus on effectiveness does limit the 
usefulness of performance measurement. For example, in Dayton little 
program evaluation is undertaken to develop new, more efficient methods 
of providing various services, although Dayton officials claim that this is 
occurring naturally as a result of MBO. Special program evaluations are 
undertaken when major problem areas are identified but limited resources 
prevent evaluations of all departments on a systematic basis. 


Dayton's performance measurement information system is basically 
manual, and there have been problems compiling performance statistics in 
a timely manner. Nevertheless, quarterly performance reviews are still 
conducted. While a computerized information system could facilitate 
more frequent reviews, the City of Dayton regards quarterly reviews as 
acceptable for performance control purposes. 


Verification of performance criteria and results suffers from the 
resource constraint policy which applies to all City operations, in that 
there are insufficient resources to conduct comprehensive performance 
audits annually. Performance audits are conducted on one-third of all 
departments on a yearly basis. The City's Office of Management and 
Budget visits various departments throughout the year and requires that 
information on the accomplishment of departmental goals and objectives 
be produced and verified. The Office of Management and Budget, in an 
attempt to cope with the resource constrant policy, has recently been 
conducting performance audits only on a sample of the performance 
objectives out of the many objectives a department may have, on the 
theory that accuracy in the sample implies accuracy in the totality. 
Performance audits are seen as critical to the maintenance of a credible 
performance measurement program. 


CG: Thunder Bay, Ontario 


Thunder Bay is a city of 112,000 people located at the head of Lake 
Superior. The city is an internationally recognized leader in municipal 
administration, having implemented such progressive management 
techniques as zero-based budgeting, corporate planning, Municipal 


Maintenance Management System, and a CAO form of organization 
structure. 


In early 1980 the City, with the assistance of the provincial 
government of Ontario, embarked upon a performance measurement 
program. For each major service area in every department, the City has 
developed a set of workload, efficiency and effectiveness indicators by 
which performance will be judged. These measurements and a description 
of the process by which the program was implemented, are contained in 


the publication, Performance Measurement Manual and Catalogue, which 


accompanies this publication. 
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The procedure which Thunder Bay adopted for organizing a 
performance measurement program was generally that described in 
Chapter III of this publication. It is worth repeating an observation made 
earlier, though, that for Thunder Bay, the development of performance 
measurements was not contingent upon formulation of goals and 
objectives, but rather was concurrent with such formulation. The major 
lessons taught by the Thunder Bay experience are that top management 
support is absolutely essential, and that the development and 
implementation of performance measurement must be a flexible, 
participative process. 


A gap in Thunder Bay's program at the present time is the lack of 
electronic data processing capability sufficient to incorporate the 
performance measurement efforts of all municipal departments. 
Performance data is by and large collected and processed manually, and 
thus its major use is found in budget deliberations. However, Thunder Bay 
is currently developing a sophisticated Management Information System 
into which performance measurement will be integrated. Upon such 
integration, Thunder Bay's performance measurement program can be 
expected to evolve into a dynamic tool for planning, control and 
evaluation. 


D. Ottawa-Carleton, Ontario 

The Regional Municipality of Ottawa-Carleton (the Region), located 
in Canada's national capital region, serves almost 550,000 people dwelling 
in eleven constituent municipalities. The Region commenced its 
performance measurement effort in mid-1978 and is currently proceding 
with implementation. 


With a mandate from Council, the Finance Department's budget 
section led the largely in-house development effort. Responsibility for 
the performance measurement program was recently transferred to the 
Region's Management Services Department. Like Thunder Bay, Ottawa- 
Carleton has found it necessary to rely heavily upon measurement 
examples drawn from other municipalities. However, the Region has been 
more explicit than Thunder Bay in its emphasis that formulation of goals 
and objectives should precede measurement selection. Performance 
measurements must be based upon departmental goals and objectives 
which must in turn be guided by policies and goals previously set by 
Council. Although the Management Services Department plays the 
central role of co-ordinating and implementing the program, the 
formulation of proper goals, objectives and performance measurements is 
considered a shared responsibility of Council, Management Services 
Department and operating departments. An example of the performance 
indicators of one of the Region's services is presented at the end of this 
Chapter. 


The performance data is primarily used to evaluate budget 
submissions. All budgetary areas, then, are required to incorporate 
performance data into such submissions. Each area must state its 
objectives in measurable terms and must formulate an appropriate set of 
demand, workload, effectiveness and efficiency indicators. Performance 
measurement in Ottawa-Carleton has not yet developed beyond the 
budget support usage into a more frequent and systematic planning, 
control and evaluation system. 
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The Regional Municipality of Ottawa-Carleton was one of the very 
first municipalities in Canada to embark upon a_ performance 
measurement program and, as with any pioneer, it has accumulated 
experience valuable to others following the same trail. The Region found 
implementation to be somewhat slower than hoped for. The process could 
probably have been expedited to a greater degree if there had been a 
clearer enunciation and identification of strong and co-ordinated support 
to the system by top management. As well, formal training and 
orientation of participants, and the provision of analytical advice to 
operating managers did not occur until late in the process and this 
contributed to some delays. Ottawa-Carleton officials believe that such 
delays could have been avoided had the in-house implementation effort 
been augmented with experienced consulting assistance. 


Despite these implementation problems the performance measure- 
ment program in the Region of Ottawa-Carleton remains conceptually 
sound, particularly in its emphasis on demand indicators and on the 
explicit formulation of goals and objectives. 
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REGIONAL MUNICIPALITY OF OTTAWA-CARLETON 


FUNCTION Environmental Services 
PROGRAM Water Supply 
ACTIVITY Filtration Plants 


PERFORMANCE OBJECTIVE 


To provide a continuous and adequate supply of 
potable water within Ministry of Environment 


guidelines. 


PERFORMANCE INDICATORS 


DEMAND 


WORKLOAD 


PRODUCTIVITY 


EFFECTIVENESS 


Raw water conditions: 
average turbidity 
average coliform count 
average P.H. level 
No. of commercial & residential units 


Million gallons of water pumped 
Tons of alum used 

Tons of chlorine used 

Tons of fluoride used 


Cost of treatment/million gallons 
No. of gallons processed/day 
Manhours/million gallons processed 


% of water samples meeting bacterial 
standards 

% of water samples meeting P.H. standards 

% of residential & Commercial units served 

% of time water restrictions in effect 


-33- 


Vil CONCLUSION 


This publication has presented a practical guide for municipal 
officials interested in the measurement of the performance of services 
which they provide. It has discussed the basic concept of and need for 
municipal performance measurement. It has developed the concepts of 
effectiveness and efficiency and provided guidelines for the selection of a 
set of good performance measurements. It outlined how to organize a 
municipal performance measurement program. It went on to discuss the 
techniques of performance data collection and the uses of performance 
results. It presented examples of performance measurements for some 
particular services and it highlighted the experiences of four 
municipalities that have implemented performance measurement. We 
hope this will provide a good basis and encouragement to those 
progressive municipalities considering the adoption of performance 
measurement. 


Is performance measurement in your future? For most 
municipalities the answer to that question is that the quantification of 
municipal performance is probably inevitable. Most municipalities will be 
unable to deal in the traditional way with the cost/revenue squeeze of the 
1980's. Most will find that traditional revenue sources, transfer payments 
and new assessment, are shrinking in relation to the growth of old and new 
demands for increased expenditure. Municipalities, then, must become 
more effective and efficient and they will need quantitative information 
to identify and evaluate opportunities to do so. 


Performance measurement can be postponed, of course, even if no 
new revenue from senior levels of governments or new assessment is 
forthcoming. The taxation rate can be raised, and because municipalities 
have both the legal capability to raise taxes and the monopolistie control 
of publie service provision, they can compel citizens to "buy" municipal 
services. But for how long ean citizens, faced by a similar cost/revenue 
Squeeze themselves, be expected to acquiesce to increased taxes? Sooner 
or later they will demand that municipalities increase effectiveness and 
efficiency and municipalities will require the tools to do so. They will 
require performance measurement. 


Early in this publication, the point was made that performance 
measurement is not the answer to all municipal problems, but rather is a 
technique for understanding problems and identifying opportunities for 
improvement. Early in this publication, too, the point was made that the 
quantification of the evaluation process was not a substitute for political 
judgement and professional experience, but rather was a complement to 
them. Municipal politicians and administrators today are facing serious 
decisions, decisions which must be made on the basis of the best available 
information. Performance measurement will improve that information. 


It is therefore neither a panacea nor a threat. It is simply good 
management. 
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